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ABSTRACT 


This research was inspired by the need to create a universal net-centric 
environment to enable collaborative defense capabilities and deliver knowledge 
dominance to the DoD. Since superior information management and the use of 
collaborative IT technologies is fundamental to building intelligence capabilities, this 
study aims to contribute to the optimization of collaborative system use by military 
groups and organizations. 

The proposed research model illustrates and explains the direct relationships 
between collaborative IT competence and collaborative functionalities, which can be 
used not only to assess current technologies but also aid in requirements generation for 
designing the ideal collaborative tool suite. Central to the research model we introduce 
the concept of collaborative IT competence, defined as the effective use of collaborative 
functionalities, and explore its relationship to performance outcomes. 

Having pre-tested and validated the proposed research model by means of 
empirical data collection in the form of an end-user survey instrument we recommend 
further research be conducted on a Navy-wide scale to evaluate the 181 collaborative 
technology tools currently in use. End-user/warfighter insight will dramatically influence 
future CIT investment decisions by providing decision makers critical information 
regarding the pragmatic versus the advertised attributes of the application/tool suite. 
Additionally, this model is designed to provide the road map to the ideal combination of 
core functionalities and required collaborative IT competence. 
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EXECUTIVE SUMMARY 


This study examines the potential benefits of collaborative technologies on group 
performance, captured with (a) project efficiency, (b) project effectiveness, (c) 
satisfaction, (d) shared situational awareness, (e) improved decision-making, and (f) 
reduced information overload. This study originally intended to address the symptoms 
and causes of the email overload issue within DoD evolved from a request initiated by 
LCDR James Mills of the Navy Information Professional Community and is sponsored 
by Dave Wennegren, Department of the Navy, Chief Information Officer. However, e- 
mail overload is only one of the factors contributing to a much larger information 
management problem among collaborative work groups. This bigger picture problem is 
what this thesis aims to address with the aid of collaborative technologies. 

A research model was designed and pre-tested that identifies the individual and 
collective variables that influence collaborative system use. The model is composed of 
Individual and Collective Variables, which are factors of the Collaborative System in 
Use. The combination of Collaborative IT Competence and Collaborative Functionalities 
result in the five unique Group Performance Outcomes influenced directly by individual 
and system Control Variables. This study presents some of the factors, influences and 
relationships, which begin to explain the complex dynamics of collaborative system use. 

The research model was tested with a web-based survey questionnaire. The 
survey response period was seventeen days - 17 August 2004 through 3 September 2004. 
The respondents were asked to answer questions in regards to a specific named project 
that they had recently managed or participated in using collaborative technology tools. 
The subject population of primary contacts in this study totaled 39 collaborative 
technology managers and 120 Information System Technology graduate students at the 
Naval Postgraduate School. Fortunately the model proved predictive in spite of the small 
sample size and target response rate of 23%. Specifically, the results reveal that 
collaborative IT competence significantly influences all four performance measures, 
explaining a substantial amount of their variance. In turn, three predictor variables - 
perceived ease of use, computer self-efficacy, and system customization - are the primary 
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antecedents of collaborative IT competence. The research model demonstrated there was 
a strong correlation (.80) for a perceived reduction in email overload by using CIT. 

This study presents an initial step towards better understanding the individual and 
collective variables that directly influence collaborative system use. It also establishes a 
critical link and feedback loop that includes all stakeholders of the CIT environment 
process. The net result of this new interaction being decision makers make better 
informed decisions by effectively empowering every end user to influence their 
environment by being a part of the process. Other benefits include the ability to foster 
and encourage the effective use of collaborative technology as well as provide insight 
into design enhancements of future collaborative capabilities. The ultimate practical 
contribution of the final research model then becomes a direct contribution to the 
eventual CIT universal defense capability and policy. Based on this successful pre-test of 
the research model, it is recommended the research model/survey tool be used on a 
significantly larger subject population. This will allow for additional analysis of 
additional contributing factors that influence collaborative system use. 



I. 


MOTIVATION 


Today’s Navy needs everything a network-centric force offers in theory, 
acquiring the ability to fully exploit technological advances by gathering and sharing 
accurate and appropriate information, building situational awareness and achieving 
decision superiority is the ultimate goal. This thesis is aimed at better understanding 
some of the critical issues related to this effort. Specifically the proposed theory-driven 
research model is tested using a survey methodology to determine the degree to which 
Collaborative System Use influences performance outcomes. Analysis of the relationship 
between ‘collaborative IT competence’ and performance presents the ability to identify 
the factors that directly influence group productivity. Wide-scale utilization of this 
research model would enable the Navy and DoD as a whole to assess which collaborative 
functionalities and in what combination are critical to group success. 

This thesis originates from a project to address the DoN e-mail overload problem. 
However, e-mail overload is a symptom of a much larger information management 
dilemma among work group members. As capabilities in Collaborative Information 
Technology (CIT) have evolved and progressed, the ability for individuals to coordinate 
and ultimately collaborate in their efforts has become both essential. One of the many 
problems facing organizations is the question of which technologies and in what 
combination to field them to achieve the best returns. Email and its expanding misuse is 
a prime example of how one of the early tools by itself has proven to not scale well, not 
obtain the desired effects, and has lead to more confounding problems. 

The evolution of CIT capabilities is being driven by both needs and wants 
simultaneously. After the events of September 11, many organizations were driven to 
CIT solutions out of necessity to reestablish some semblance normalcy. The Navy’s 
vision for the future and the development of its Net Centric Warfare and Information 
Superiority (NCW/IS) concept are dependent on CIT working well. CIT has the potential 
for providing significant returns in costs reduction, timesavings, quality of work 
produced, and creativity. 
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We propose the definition of ‘collaborative IT competence’ and its interaction 
with several key collaborative functionalities to introduce the concept of Collaborative 
System Use as the engine that fuels specific group performance outcomes. By improving 
the quality and speed with which members of an organization can exchange information 
there will be an improvement in the overall knowledge management environment. Given 
the highly desirable benefits of inter-disciplinary collaboration and team decision¬ 
making, this study aims to shed light on the driving factors within collaborative 
technologies on group performance. 

A. RESEARCH QUESTIONS 

Relevant literature concerning collaborative information technology supports the 
argument that defense groups can enhance their mission performance through 
collaborative technologies and can provide seamless integration among group members 
to assure that correct, timely, and protected information to the right person anywhere, 
anytime. These collaborative capabilities can help enhance warfighter situational 
awareness, ensure information access, optimize knowledge resources, build integrated 
solutions, and adopt best practices across groups (NWDC IKA 2004). The existing 
research supports the efforts to establish a baseline understanding of the productivity and 
efficiency gains to be realized through the use of collaborative technology. The 
following research questions are designed to gather and facilitate the understanding of 
current collaborative technology tools/applications in use DoN-wide. 

• Whether, how, and why collaborative technologies can improve group 
performance outcomes (e.g. efficiency, effectiveness, satisfaction, 
situational awareness, reduction in email overload)? 

• How do organizations achieve the potential benefits of collaborative 
technologies? 

• What individual and group factors drive the effective use of collaborative 
technologies? 

B. RESEARCH MODEL 

The following model will be used to address the research questions. Specifically 

the research model provides a proof of concept methodology for determining how 

Collaborative IT Competence can facilitate performance benefits. The model is 

composed of Individual and Collective Variables, which are factors of the Collaborative 

System in Use. The combination of Collaborative IT Competence and Collaborative 
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Functionalities result in the five unique Group Performance Outcomes influenced directly 
by individual and system Control Variables. 
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Figure 1. Research Model 


Group Performance 

Process Efficiency 
Process Effectiveness 
Group Satisfaction 
Situation Awareness 
Reduce Email Overload 


Control 

Variables 


3 












































THIS PAGE INTENTIONALLY LEET BLANK 


4 



II. THEORY DEVELOPMENT 


A. LITERATURE REVIEW 

The ubiquity of the Internet’s connectivity brings together widely dispersed 
entities to enable collaboration in places where it was not feasible before, resulting in the 
emergence of ‘online collaborative work’, which has transformed the established nature 
of traditional collaborative work and sparked increased academic and practical interest 
(e.g., Easley, Devaraj, and Grant, 2003). 

Sophisticated collaborative technologies, such as Groove Networks and Oracle 
Collaboration Suite are a relatively recent development. These technologies are 
integrated sets of collaborative functionalities that enable synchronous and asynchronous 
communication and information sharing among inter-connected entities. Despite the 
widely publicized potential of these collaborative technologies to develop dynamic 
capabilities to adapt to information-intensive environments, we still know little whether, 
how, and why these technologies can enhance group productivity (e.g., project efficiency 
and effectiveness, situational awareness, group satisfaction, email overload). 

B. COLLABORATIVE WORK 

Collaborative work is considered the foundation of any organization’s success. 
Therefore, research on collaborative work and its impact on team and organizational 
performance has been prominent. Recent IT advances have enabled collaborative systems 
that aim to facilitate and support collaborative work. Evidence suggests that these 
technologies are widely adopted in practice. Nevertheless, we still know little whether, 
how, and why collaborative technologies can enhance group capabilities and 
performance. 

C. COLLABORATIVE TECHNOLOGIES 

Collaborative technologies are integrated set of IT-enabled functionalities that 
enable synchronous and asynchronous communication and information sharing among 
inter-connected entities from virtually any geographical location. Today’s collaborative 
technologies are more sophisticated versions of computer-supported ‘Group 
Communication Support Systems’, ‘Group Decision Support Systems’, or ‘groupware’, 
which are computer aids designed to support collaborative work. 
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While there is a long history of collaborative technologies (Licklider and Taylor, 
1968, Nunamaker, Dennis, and Valacich, 1991), Internet-based technologies are 
becoming the most prominent in practice, especially among dispersed groups (Wheeler, 
Dennis, and Press, 1999). Effective use of collaborative technologies enables groups to 
communicate, collaborate, and interact, facilitating dispersed interaction across time and 
space (Sole and Applegate, 2000). 

1. Functionalities in Collaborative Technologies 

Common collaborative functionalities include email/messaging, calendar, file¬ 
sharing, voice and audio conferencing, whiteboards and shared workspaces, and smart 
search, among others. If these functionalities are effectively used, they can support 
brainstorming and sharing of new ideas (Hamilton 2001, Molloy and Schwenk, 1995), 
constructing project histories (Grantham and Nichols, 1993), and reaching group 
consensus (Rockart and Short, 1989), among other positive outcomes. The effective use 
of these key functionalities jointly determines the group’s ‘collaborative IT competence’. 
D. COLLABORATIVE IT COMPETENCE 

Since the IT artifact is still not well-defined (Orlikowski and lacono, 2001), to 
better understand the impact of IT on dynamic capabilities, we propose the concept of 
‘collaborative IT competence’. Following Sambamurthy et ah, (2003), ‘collaborative IT 
competence’ is defined as ‘the ability to acquire, deploy, and leverage collaborative 
functionalities in combination with other resources to support collaborative activities in 
value adding ways’. Other resources include, but are not limited to human, managerial, 
and organizational resources, plus other IT functionalities and technologies. We propose 
the combination of the use of Collaborative Functionalities and Collaborative IT 
Competence will have positive influence on five performance outcomes; Process 
Efficiency, Process Effectiveness, Group Satisfaction, Situational Awareness, and 
Reduction in Email Overload. 

Collaborative IT competence is distinguished from related constructs (King, 
2002). First, IT competence captures effective management of collaborative IT 
functionalities, not merely their existence (Wiseman, 1988). Second, IT investments do 
not necessitate that proper collaborative IT systems will be acquired, integrated, and 
managed (Feeny and Willcocks, 1998). 
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IT competence has been generally a multi-dimensional construct (Bharadwaj, 
2000). For instance, Ross et ah, (1996) argue that IT competence is developed along 
multiple dimensions. Feeny and Willcocks (1998) also propose nine IT functionalities 
for business success. Collaborative IT competence is composed of seven representative 
(non-exhaustive) facets (Table 1). 


Effective Use of Collaborative Functionalities 

Effective Use of Email Functionality 

Email to exchange messages among team members. 

Effective Use of Chat/Instant Messaging Functionality 

Chat/Instant Messaging to share information in real-time. 

Effective Use of Calendar Functionality 

Calendar for connecting time and location information for all team members 

Scheduling for providing up-to-date calendar information. 

Effective Use of File-Sharing Functionality 

File sharing to store, archive, and reuse information and best practices. 

Consolidation and synchronization of files into a single repository for easy access. 

Effective Use of Conferencing Functionality 

Conferencing for spawning new ideas and solutions. 

Collaboration among team members to interact in real time. 

Effective Use of Whiteboard/Shared Workspace Functionality 

Shared workspace for simultaneously working together in real-time. 

Whiteboard functionality for bringing together team members. 

Effective Use of ‘Smart’ Search Functionality 

Search functionality for quickly locating information and files. 

Search functionality for enabling quick access to content. 


Table 1. Operationalization of Collaborative IT Competence 


Following the notion of ‘product convergence’ (Mantena and Sundarajan, 2002), 
these seven functionalities are more effective when all of them are simultaneously used 
due to their complementarities (Kraut and Streeter, 1995). Collaborative IT competence 
captures the inter-relationships among these functionalities under a parsimonious 
construct to explain their joint influence on collaborative processes. 

E. IT COMPETENCE & GROUP PERFORMANCE 

Collaborative ITC will allow organizations to complete tasks better, faster, and 
cheaper. For the military increased collaborative ITC will be a key enabler for generating 
and sustaining combat power and controlling the operational tempo in the future network 
centric-warfare environment. Process efficiency consists of project time and 
development cost (Kusunoki et al, 1998). Second, product effectiveness consists of 

project quality and innovativeness. Finally, situation awareness reflects the group’s 
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ability to understand their environment and enhance collective decision making. Through 
effective coordination, learning, and collective mind with the aid of collaborative IT 
competence, groups can enhance these performance outcomes. In sum, collaborative IT 
competence enhances group innovation, flexibility, responsiveness, and collaboration 
(McGrath and lansiti, 1998). 

1. Process Efficiency; CIT Enabling Faster and Cheaper Processes 

CKE Restaurants, the parent company of Carl’s Jr., Hardee’s, La Salsa, Green 
Burrito and Timber Lodge Steakhouse restaurants expects to be able to reduce the 
amount of time spent on creation and editing corporate training materials by more than 
fifty percent (Communication News, 2003). Proctor and Gamble by using CIT and 
alternative workplace arrangements aimed to reduce its real estate costs by $300 million 
annually (Harmon, 2001). In a military setting when a task force embarked on board 
naval shipping begins a rapid mission-planning scenario significant amounts of time and 
effort is expended just trying to centrally locate all of the key planners from within the 
task force on a single ship. With the use of CIT these key planners can get their mission 
specific orders and begin planning immediately, allowing for a jump-start of the process 
in a time critical scenario. 

2. Process Effectiveness; When Being Geographically Collocated is Not 
an Option 

In the case of Morgan Stanley, the largest tenant in tower number two of the 

World Trade Center, they employed over three thousand people on multiple floors. When 

the towers collapsed, they were faced with two situations that required their immediate 

attention. First, how to reestablish functional operations to temporarily carry the burden 

of the offices lost in the towers collapse. Second, once they were able to reestablish local 

offices in the Manhattan area connecting various entities in the greater Manhattan area 

that were no longer centrally located in a single building. Morgan Stanley claims they 

were operational within 48 hours and they achieved this result by relocating people to 

backup facilities in New Jersey and Brooklyn (Tully, 2001). One of the core principles 

for survival on the battlefield is dispersion. Commanders constantly face the challenge of 

balancing the need for dispersion; to minimize the effects of any one of the enemy’s 

weapons systems, and the need to mass ones own forces to achieve desired outcomes. By 

using CIT, modern commanders have the ability to exercise maximum dispersion 
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physically yet be virtually collocated for planning and coordination allowing for the 
actual massing of forces at the decisive time of action. 

3. Group Satisfaction; Unique Solutions to Traditional Challenges 

The perception that air travel is a stable, secure, reliable transportation system was 
given a significant jolt on September 11, 2001. Air travel was and is still one of the 
safest modes of transportation on the face of the earth but the world model has been 
subject to some significant adjustment in the aftermath of the September 11 attacks. 
With the changes in security screening, vehicle procedures at airports, curbside check of 
baggage, reductions in numbers of flights, air travel became significantly less convenient. 
With the significant eroding of the public confidence in the safety of the airline industry 
and its ability to prevent a repeat of the events September 11^*’, many organizations have 
been looking for alternative solutions to the traditional in person business meeting. CIT 
has presented solutions that have a unique and significant positive impact on the 
perceptions on the group as a whole. 

4. Situational Awareness; Higher Degree of Knowledge Sharing 

The ability for multiple members of a work group to have an up to date vision of 
the state of a project can is significantly enhanced by CIT. With file sharing and web 
conferencing groups can spread out over a large geographic area can simultaneously 
download, view, edit and discuss project documents or presentations. All of the team 
members are able to update their vision of the projects status and rapidly resolve any 
ambiguities or areas of contention in a real time environment. 

5. Organizational Performance; Increases in Creativity 

Design teams from multiple organizations are able to brainstorm and coordinate 
their efforts from hundreds to thousands of miles away. In addition since problems are 
not necessarily limited to specific disciplines or schools of thought, new and creative 
perspectives can be brought to the table that otherwise might have been ignored, 
overlooked, or never sought out. 

F. IMPROVING THE PERCEIVED EFFECTIVENESS OF 

COLLABORATIVE TECHNOLOGIES 

Having shown the impact of collaborative technologies on group performance, the 
next step aims to prescribe how their effective use can be enhanced by understanding 

which individual and collective factors drive the effective use of collaborative systems. 
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Following an extensive literature review we identified a group of factors that drive the 
effective use of collaborative technologies. 

The rationale for the antecedents of collaborative IT competence is based on 
Ajzen’s (1985, 1988, 1991) Theory of Planned Behavior (TPB), which has been one of 
the most influential theories in explaining and predicting human behavior. According to 
TPB, the proximal determinant of a behavior is a behavioral intention - after all, people 
do what they intend to do. Behavioral intentions are motivational factors of how hard 
people are willing to try and how much of an effort they are planning to exert to perform 
a behavior (Ajzen 1991, p. 181). Behavioral intention, in turn, is determined by three 
factors: perceived attitude toward the behavior, perceived subjective norm (SN), and 
perceived behavioral control (PBC). Attitude captures the overall evaluation of the 
desirability of the behavior, SN refers to the expectations of important others with regard 
to the behavior, and PBC refers to the ease or difficulty of performing the behavior. The 
antecedents of attitude, SN, and PBC are a set of underlying altitudinal beliefs, normative 
beliefs, and control beliefs, respectively. 

1. Individual IT antecedents 

Following Venkatesh et ah, (2003), individual acceptance and use of IT is driven 
by four factors: 

a. Perceived Usefulness 

Perceived usefulness is defined as the extent to which a system user 
believes that using a system will enhance her job performance (Davis, 1989). It enables 
accomplishing tasks more quickly, improving job performance, and increasing 
productivity. Perceived usefulness improves a user’s attitude toward using a collaborative 
system, thus facilitating its effective use. 

b. Perceived Ease of Use 

Perceived ease of use or system quality is defined as the extent to which a 
system user believes that using the system will be effortless (Davis, 1989). It reflects 
whether a system is clear and understandable, intuitive, and does not require a lot of my 
mental effort. Perceived ease of use also improves a user’s attitude toward using a 
collaborative system, also facilitating its effective use. 
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c. Computer Self-Efficacy 

Self-efficacy generally reflects individual judgments of a user’s 
capabilities to perform a behavior (Bandura, 1997). Computer self-efficacy reflects the 
extent to which a user can use a system with minimal assistance (Compeau and Higgins, 
1995). If a user can complete a job using a collaborative system with no or minimal 
assistance, he is more likely to use the system effectively. 

d. Facilitating Conditions 

Facilitating conditions or controllability reflects individual judgments 
about the availability of resources and opportunities to perform a behavior (Ajzen, 1991). 

Applied to collaborative technologies, it reflects whether a user has the knowledge, 
resources, and opportunity to use a system. Facilitating conditions thus enables a user to 
use a collaborative system effectively. 

2. Collective IT Antecedents 

In addition to individual factors, given the group nature of using collaborative 
technologies, several other collective factors are likely to facilitate the effective use of 
collaborative systems. 

a. Trust in Interpersonal Trustworthiness 

Intra-group trust reflects the extent to which group members trust each 
other. Trust also captures whether promises to each other are reliable, whether group 
members are honest to each other, and whether they would go out of their way to help 
each other. Trust improves the attitude toward a behavior, thus facilitating groups to use a 
collaborative system collectively. 

b. Information Sharing 

Information sharing reflects the extent of information and knowledge 
exchange within group members. It captures satisfaction with the knowledge exchanged 
within the group and comfort with sharing sensitive information with group members. If 
groups openly share information, they are more likely to use collaborative technologies 
effectively. 

c. Social Cohesion 

Social cohesion represents whether group members are friendly and have 
good interpersonal relationships (Sethi et ah, 2001). Social cohesion enables group 
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members to work together well without interpersonal conflicts, and it thus allows groups 
to effectively use a collaborative system. 
d. Social Norm 

Social or subjective norm refers to the judgments of important others with 
regard to a behavior. It also reflects whether important others actually undertake the 
behavior (Ajzen, 1991). Following Ajzen, the higher the social norm, the more likely 
groups will use a collaborative system. 

G. CONTROL VARIABLES 

System Voluntariness has to do with the level of mandatory use of the technology 
in order to complete ones assigned tasks as (Venkatesh, 2000). Groups who are forced to 
use a particular system may behave differently from those who have freedom in choosing 
to use it or not. While initial utilization may start higher, creativity may be repressed and 
resentment toward forced change is a likely outcome 

Personal Habits for the use of the system has to do with the willingness of users to 
make it part of the users regular routine (Limayem and Hirt, 2003). Habit can be one of 
the factors driving the use of collaborative technologies beyond the proposed IT 
antecedent factors. 

In addition, previous training on the use of various tools and system 
customization to a specific task were indicated by the review of the literature as being 
meaningful to the use of IT. We expect superior training to result in higher collaborative 
IT competence and group performance. 

System customization could also have an influence on IT competence by allowing 
users more in depth input to how the system performs specific tasks. The ability to take a 
off the shelf software package and customize it to an organizations particular needs could 
be significant to ITC. 
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III. RESEARCH METHODOLOGY 


A. SURVEY ADMINISTRATION 

Research was primarily conducted on collaborative technologies used within 
DoD. Given the volume of collaborative technology applications currently in use, only 
seven core collaboration functionalities were selected for analysis. The selected core 
functionalities were based on input of a DoD-wide consensus as reported by Sahar 
Tamen July 2004. The seven functionalities; email, chat/instant messaging, calendar, 
file-sharing, web conferencing, whiteboard/shared workspace, 'smart' search functionality 
were in turn studied to determine their direct relationship to enhancing collaboration, 
group productivity, and situational awareness amongst group members. 

We used ‘key informant’ methodology relying on a single participant to offer 
information about a collective setting user perspective to establish connections between 
specific collaborative tool functionality and user productivity. The subjects for this data 
collection are DON collaborative technology stakeholders and users. These individuals 
were identified as key managers of government and industry collaborative technology 
users. Each was initially contacted based on their known use of specific collaborative 
tools. The subject population expanded as primary points of contacts referred other 
known managers and users. 

The population of primary contacts in this study totaled 39 collaborative 
technology managers and 120 Information System Technology graduate students at the 
Naval Postgraduate School. The survey was completed by 57 respondents and 37 
collaborative technology users - a total response rate of 36%, however only a 23% 
effective response rate. 

Upon launch of the survey each manager was contacted via email and instructed 
that a participant invitation survey would follow shortly. Each manger was asked to 
preview and take the survey then forward the participant survey invitation via their 
distribution channels to collaborative technology users within and working with their 
organization. Eollow up contact was made one week after launch of the survey via email 
and phone calls. Eollow up phone calls revealed many managers did not distribute the 


13 



survey as requested. Per Department of Defense Instruction Number 1100.13, we were 
not permitted to mass distribute this survey instrument to other DoD components.! 

As the survey content was web-based and access was acquired via a URL link 
provided in the email invitation to participate in the study. The survey was broken into 
six sections. In section one of the survey respondents are asked to identify a typical (not 
the most or least successful) project that they recently managed and answer the survey 
questions relative to that experience. The following identifiers were requested from each 
individual: project title, number of individuals in group, and position within group. 
Sections two through five contained all measurement items and the last section of the 
survey consists of six questions requesting general demographic information about the 
respondent. Additionally, the respondents were provided a text field for additional 
comments and each respondent was asked if they desired receive an executive summary 
of the results and the thesis resulting from this study, if so they were required to provide 
their email. The purpose of acquiring this professional information is to categorize the 
respondent population. 

The data was collected by measurement items based on a series of questions 
formatted for check box, rating button, and short answer responses. The survey response 
period was seventeen days - 17 August 2004 through 3 September 2004. The goal was 
to survey individuals who had utilized collaborative technology in a group setting; those 
that were not users were directed to the end of the survey and thanked for their time. 

B. MEASUREMENT DEVELOPMENT 

Wherever possible, measurement items were adapted from existing scales. For 
new measures and for those that required significant deviations, standard scale 
development procedures were used (Straub, 1989). First, the domain of each construct 
was specified. Second, a large pool of items was developed based on the conceptual 
definition, assuring that these items tap the construct’s domain. From this pool, items 
were chosen based on whether they conveyed different, yet related shades of meaning 

1 The distribution of this survey was limited by Department of Defense, 
INSTRUCTION NUMBER 1100.13, Dated: November 21, 1996, SUBJECT: Surveys of 
DoD Personnel. This Instruction governs all surveys of DoD personnel, members of the 
Military Services and their families, as well as surveys conducted by the DoD 
Components of personnel in other Eederal Agencies and members of the public, when the 
results are to be used for general statistical purposes. 
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(Churchill, 1979). Aside from control variable and demographic data questions, all 
questions were structured on a ‘Liker-type’ scale: one (lowest) to seven (highest). 
Measurement items are listed in Appendix A. 

Collaborative IT Competence: Collaborative IT competence was measured based 
on a review of the literature that aimed to assess the extent to which group members 
effectively use IT functionalities. Special care was taken to tightly link IT functionality 
with specific processes (Lind and Zmud, 1995). The effective use Effective Use of the 
Collaborative System was measured with twelve newly developed items following the 
work of (Pavlou and El Sawy, 2004). 

Performance Indicators: Project Efficiency and Project Effectiveness were 
measured with two items each both based on (Kusunoki et ah, 1998). Situational 
Awareness was measured with three items based on (Endsley, 1996). Satisfaction was 
measured with two items based on (Rai, Eang, and Welker, 2002). Collaborative System 
Use was measured with five items specifically designed for this study to gather 
information regarding individual behavior patterns with relation to collaborative system 
use. Perceived Usefulness was measured with three items, based on (Venkatesh, 2000). 
Ease of Use was measured with three items, following (Rai et ah, 2002; Venkatesh, 
2000). Computer Self-Efficacy was measured with five items, based on (Compeau and 
Higgins, 1995). Eacilitating Conditions was measured with two items, based on 
(Venkatesh, 2000). 

Group Dynamics: Beliefs in Interpersonal Trustworthiness was measured with 
four items, based on (Jap, 1999). Information Sharing was measured with two items, 
based on (Beccara-Eernandez and Sabherwal 2001; Bensaou and Venkatraman, 1995). 
Social Cohesion was measured with two items, based on (Sethi et ah, 2001). Social 
Norm was measured with two items, based on (Ajzen, 1991) and Environmental 
Uncertainty was measured with one item, based on (Pavlou and El Sawy, 2004b). 
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IV. RESULTS AND DISCUSSION 


We used the Partial Least Square (PLS) method to analyze our data. PLS employs 
a component-based approach for estimation purposes (e.g., Lohmoller, 1989), and places 
minimal restrictions on measurement scales, sample size, and residual distributions (Chin 
et al. 2003). In general, PLS is better suited for explaining complex relationships as it 
avoids two serious problems: inadmissible solutions and factor indeterminacy (Fomell 
and Bookstein, 1982). This sentiment is further echoed in (Wold, 1990), who writes: “In 
large, complex models with latent variables PLS is virtually without competition.” (p. 
590). In sum, we chose the PLS method in order to accommodate the large number of 
principal constructs in our model. 

A. DESCRIPTIVE STATISTICS 

Demographic information is shown in Table 2 (group and respondent 
characteristics). 


Variables 

Group 

Size 

Group Experience with 
collaborative functionalities 
(months) 

Gender 
(% males) 

Age 

(years) 

Education 

Work 

Experience 

(years) 


iiisirois) 

13 1 (17.4) 

8S(32/ 

37.1 (8 5) 

SoTie College 

16 1 (S.3) 


Table 2. Group and Respondent Demographic Characteristics 


About half of the respondent groups used Groove (27%) and Navy Knowledge 
(23%). Other collaborative technologies used included Oracle (8%) and Autonomy (5%). 
However, 29% of the respondents reported using more than one packages. 

Forty-five percent of the key informants noted their role as leader or manager, and 
55% as member or participant. Their key functions were military (40.5%), Information 
Technology (38%), Sales (8%), and Engineering (13.5%). In terms of training received, 
64% received no formal training, 18% received classroom training, 12% received hands- 
on training, and 6% received online training. Finally, the respondents self-reported level 
of experience was novice (22%), intermediate (46%), advanced (22%), and expert (5%). 
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The total number of completed usable responses was 37 out of the 122 
respondents we contacted (35% response rate). Non-response bias was assessed by 
verifying that early and late respondents were not significantly different (Armstrong and 
Overton, 1977). The first set of tests compared gender, age, education, education, and 
work experience. The second set of tests compared group characteristics (size and 
experience with collaborative technology). All t-test comparisons between the means of 
the two groups in both sets of tests showed insignificant differences (p<0.1 level). 

Descriptive statistics and the correlations among the principal constructs are 
shown in Table 3. 




Mean 

iinil 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

mm 

o 

1 

Group 

Effectiveness 

5.68 

1.44 

.93 












2 

Situational 

Awareness 

5,34 

1.91 

,823" 

.85 











3 

tr-ail 

Overload 

4,45 

1.72 

.651" 

.669” 

.92 










4 

Collaborative 

CoTicetence 

4.39 

3.23 

782" 

.724" 

.652" 

.94 









5 

Perceived 

Usefulness 

528 

2.83 

.554" 

.588" 

.736" 

559" 

.98 








6 

Perceived 
Ease of Use 

5.10 

2.86 

,562" 

.602" 

.685" 

590" 

.585” 

.90 







7 


4.96 

iliW 

.518" 

.605" 

.595" 

547" 

483" 

.496" 

.88 






8 

Intragroup 

Trust 

5.83 

1.48 

.590" 

352* 

.506" 

522" 

.460" 

.408* 

324 

.88 







5.47 

2.20 

.787" 

.651" 

.623" 

654" 

.562" 

.631" 

501" 

.817" 

.94 






5,76 

1.62 

711* 

.498" 

.540" 

527" 

.408* 

.460" 

342* 

.838" 

.890" 




11 

Sjc-ective 

Norm 

4,78 

1.07 

324 

.208 

.085 

335* 

.033 

139 

.267 

.282 

399* 

.281 

.67 


12 

System 

Customization 

4.50 

3.03 

.657" 

449" 

.547" 

649" 

452" 

.443" 

.210 

.544" 

.568" 

.487" 

.306 

m 

** Significant at p<.01; Significant at p<.05 - Internal Consistency CoeKicients shown in bold 


Table 3. Correlation Matrix and Average Variance Extracted for Principal Constructs 

(Getting Info) 


B. MEASUREMENT VALIDATION 

Measure reliability was assessed using internal consistency scores, calculated by 
the composite reliability scores (Werts, Linn, and Joreskog, 1974).2 Internal 
consistencies of all variables are considered acceptable since they exceed .70, signifying 
tolerable reliability. Convergent and discriminant validity is inferred when the PLS 
indicators (a) load much higher on their hypothesized factor than on other factors (own- 

2 The composite reliability score is: (Zkl)2 / [(Zkl)2+ ZiVar(8l)], where A.I is the indicator loading, 
and Var(8l)=Tkl2. 
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loadings are higher than cross-loadings), and (b) when the square root of each construct’s 
Average Variance Extracted (AVE) is larger than its correlations with other constructs 
(the average variance shared between the construct and its indicators is larger than the 
variance shared between the construct and other constructs (Chin, 1998).The square root 
of all AVEs are above 0.80, which are much larger than all the cross-correlations. These 
tests suggest that all measures have adequate convergent and discriminant validity. 
Common method bias was assessed using Harman’s one-factor test (Podsakoff and 
Organ, 1986). Each principal construct explains roughly equal variance (omitted for 
brevity), indicating that our data do not suffer from high common method variance. 
Einally, multicollinearity among the external beliefs was not a serious concern since all 
checks (eigen analysis, tolerance values, VIE) did not indicate any problem. 

C. THE STRUCTURAL MODEL 

The PES path coefficients (which can be interpreted as standardized regression 
coefficients) are shown in Eigure 2. Eor clearer exposition, the item loadings of each 
construct are omitted since they are all above 0.80. All control variables were initially 
included in the model, but since none were significant, all controls were dropped. 
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COLLECTIVE VARIABLES 


Figure 2. PLS Results (n=37) 

As shown in Figure 2, collaborative IT competence significantly influences all 
four performance measures, explaining a substantial amount of their variance. In turn, 
three predictor variables - perceived ease of use, computer self-efficacy, and system 
customization - influence collaborative IT competence. 

D. DISCUSSION 

While we propose eight influencing factors to IT competence, and only three 
prove to be statistically significant, much of this can be assessed to the limited sample 
size. There is directional importance to the other variables that a large sample size could 
provide further evidence to their importance. This study presents an initial step towards 
better understanding the individual and collective variables that directly influence 
collaborative system use. It also establishes a critical link and feedback loop that 
includes all stakeholders of the CIT environment process. The net result of this new 
interaction being decision makers make better informed decisions by effectively 
empowering every end user to influence their environment by being a part of the process. 
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Other benefits include the ability to foster and encourage the effective use of 
collaborative technology as well as provide insight into design enhancements of future 
collaborative capabilities. The ultimate practical contribution of the final research model 
then becomes a direct contribution to the eventual CIT universal defense capability and 
policy. Based on this successful pre-test of the research model, it is recommended the 
research model/survey tool be used on a significantly larger subject population. This will 
allow for additional analysis of additional contributing factors that influence collaborative 
system use. 


21 



THIS PAGE INTENTIONALLY LEET BLANK 


22 



V. RECOMMENDATIONS 


This research originates out of an initiative to address e-mail overload, however 
the resultant validated research model illustrates the complexities of the core 
collaborative functionalities. A better understanding of the critical relationship between 
Collaborative IT Competence and collaborative functionalities will allow us to influence 
performance outcomes positively, including reductions in email overload. Although a 
great deal of research remains to understand and optimize Collaborative IT Competence 
this preliminary research indicates the use of collaborative tools can increase group 
productivity, cut down on the amount of extraneous e-mail traffic, and enhance 
situational awareness and decision-making. 

A. RECOMMENDATIONS FOR THE NAVY 

The Navy and DoD have spent significant amounts of time and money on what 
could be termed exploration in CIT. Unfortunately, to this point it has mostly been 
exploration without good documentations as to the goals and benefits of the use of CIT. 
Organizations throughout the Navy have deployed various CIT applications assuming 
they will reap benefits from these implementations without bothering to evaluate what the 
benefits are going to be. The assumption is things will just be better. While the ability to 
collaborate virtually through the use IT is one of the pillars for success of the Navy’s 
NCW/IS vision, little has been done to actually structure how this capability will be 
implemented Navy wide. If the Navy and DoD are truly going to achieve the desired 
benefits of CIT as envisioned in NCW/IS several key functions need to take place. 

Collaboration will continue to influence every level of military operations and in 
the future will require integration of joint, inter-agency and coalition partners. Technical 
interoperability alone will not produce a synchronized and effective fighting force. Key 
allies, other services and government agencies must be included in the development 
process of Navy network-centric capabilities (NWDC IKA 2004). To this end, step one 
requires a vigorous program of iterative concept development and experimentation, 
starting with a complete Navy inventory and assessment of collaborative tools currently 
in use. A DoN supported study utilizing the research model presented is recommended to 


empower decision makers with an accurate field assessment based on end-user input. 
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Based on the results of step one the Navy needs to stop allowing independent 
activities to choose their own solutions and an enterprise wide solution needs to be 
adopted with an emphasis on successful implementation of the core functionalities. This 
would allow entities throughout the Navy to share information across operational and 
organizational boundaries. The methodology should be based on the full understanding 
of Collaborative System Use and be specifically targeted at optimizing the utilization of 
each functionality. 

B. GROUP MANAGERS AND USERS 

Training and support on the use of the tools needs to be both robust and 
interactive in the form of hands on between the users and the developers of the tools we 
deploy. While many users will find the new tools at their disposal very intuitive and easy 
to use there may be some who are resistant to them because they involve a change in the 
way they have conducted their actions in the past. To merely run a system out to the fleet 
users and allow them to learn by doing without some form of formal training and ongoing 
support would be irresponsible both fiscally and qualitatively. Without the appropriate 
level of technical support, many of the users will likely never realize the full potential 
offered by the new tools within the realm of CIT. 

C. DOD IN GENERAL 

In the ongoing war against terrorism, one of the primary areas of interest revolves 
around the need to coordinate efforts in the realm of intelligence collection and analysis. 
By infusing interoperable collaborative technologies into all DoD communities, DoD can 
ensure integration of intelligence efforts and develop enterprise-wide capabilities. By 
developing a secure, seamless, and interoperable collaborative infrastructure and enabling 
end-to-end connectivity among all DoD departments, cross-departmental knowledge 
sharing can take place that would enable sound decisions to be made with up-to-date 
information, irrespective of rapid changes in the environment. 

D. IMPLICATIONS FOR REDUCTION IN EMAIL OVERLOAD 

Email because it has become the tool of choice for so many people and it is so 
widely accepted, has been adopted for use to a number of environments that it is not 
ideally suited. Many traditional face-to-face meetings or phone conversations have been 
replaced by email because of the perceived ease of use. One problem is once the email is 
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fired out the user looses control of the schedule of response. In a dynamic environment, 
the sender is now bound to his or her computer waiting for a response if the question is 
time critical. In addition, they have no vision as to when the message will be viewed or 
that receiver interprets the context of the message correctly. The reality is there are a 
number of CIT solutions that have been developed that provide a much richer 
information exchange and better outcomes then the use of email. As demonstrated by the 
research model there was a strong correlation (.80) for a perceived reduction in email 
overload. Just as there were those who were resistant to the idea of using email to begin 
with, there will be those who are resistant to the use and deployment of the new 
collaborative tools. 

The way organizations implement their email systems can be enhanced by the 
incorporation of other CIT technologies. Based on a case study implementation plan 
involving the use of CIT the cost savings and increases to productivity may be significant 
(Housel and Cook 2004). Oracle’s Collaboration Suite and their integrated storage 
repository for all emails in illustration how centralized storage may be beneficial. Where 
before if a user wanted to keep a copy of a particular email that message was associated 
with his particular account. If ten members of an organization wanted to keep a message, 
ten individual copies were stored on the organizations mail servers. With Oracles system 
one copy is kept, centrally stored, and then accessed by those who require it. This 
reduces the overall storage requirement for the organization and allows for better version 
control, tracking, and potentially access control enterprise wide. 

F. FURTHER RESEARCH 

The Navy and DoD have invested significant amounts time and money into 
collaborative tool applications but unfortunately have failed to tap their most valuable 
resources for performance feedback on these applications. This study presents an initial 
step towards filling that void. Specifically this research has broad implications for 
understanding the individual and collective variables that directly influence collaborative 
system use. This understanding offers many potential benefits including the ability to 
foster and encourage the effective use of collaborative technology as well as provide 
insight into design enhancements of future collaborative capabilities. The research model 
makes a practical contribution towards the eventual CIT universal defense capability and 
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policy. Given the Navy’s express interest in CIT, based on its vision the Navy of the 
future, continued research and development of this methodology is essential towards the 
ultimate goal of true net centric warfare. 
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APPENDIX. MEASURED ITEMS 


SURVEY MEASUREMENT ITEMS 

PRO) ECT CHARACTERISTICS 

Project Title: 

# 1 ndividuals in Group. 

Your Position in the Group: 

USE OF COLLABORATIVE SYSTEM (1: stronqiv disagree/ 7: strongly agree) 

email functionality 

chat/messaging functionality 

calendar functionality 

file-sharing functionality 

web conferencing functionality 

whiteboard/shared workspace functionality 

'smart' search functionality 

COLLABORATIVE IT COMPETENCE (1: strongly disagree/ 7: strongly agree) 

Email to exchange messages among team members. 

Chat/Instant Messaging to share information in real-time. 

Calendar for connecting time and location information for all team members 

Scheduling for providing up-to-date calendar information. 

File sharing to store, archive, and reuse information and best practices. 

Consolidation and synchronization of files into a single repository for easy access. 

Conferencing for spawning new ideas and solutions. 

Collaboration among team members to interact in real time. 

Shared workspace for simultaneously working together in real-time. 

Whiteboard functionality for bringing together team members. 

Search functionality for quickly locating information and files. 

Search functionality for enabling quick access to content. 

PERFORMANCE 1NDICATORS (1: strongly disagree/ 7: strongly agree) 

Project Efficiency 

We were able to meet our project timeline deliverables. 

We efficiently managed our daily workflow. 

Project Effectiveness 

Our project deliverables were of high quality. 

Improvements in quality of group's activities. 

Situational Awareness 

Do you have any difficulty finding required items of information? 

Do you have a coherent mental picture and good understanding of your project status? 

Do you have the feeling that you are able to anticipate problems? 

Satisfaction 

Your situational awareness. 

Satisfaction with the system. 

Collatmrative System Use 

1 send fewer emails. 

1 make fewer phone calls. 

1 use 1 M/chat features instead of sending email. 

1 hold/attend less meetings in-person. 

1 email less documents/attachments to group members. 

Perceived Usefulness 

Using the system would enable me to accomplish tasks quicker. 

Using the system would improve my job performance 

Using the system in my job would increase my productivity. 

Ease of Use 

My interaction with the system is clear and understandable. 

Learning to use the system was intuitive and did not require a lot of my mental effort. 
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SURVEY MEASUREMENT ITEMS 

1 find the system to be easy to use. 

Computer Self-Efficacy 

1 could complete my job using the system... 

Without assistance. 

With on-site assistance initially. 

With software manuals for reference. 

With assistance or help utility available. 

Facilitating Conditions 

1 have the resources necessary to use the system. 

With the proper training and technical support 1 would be able to use the Collaborative System. 

GROUP DYNAMICS 

Beliefs in 1 nterpersonal Trustworthiness 

We trust each other. 

Our promises to each other are reliable. 

We are honest in dealing with each other. 

We would go out of our way to help each other out. 

1 nfoimation Sharing 

1 am satisfied with the information and knowledge sharing among group members. 

1 am comfortable sharing sensitive information within my group. 

Social Cohesion 

Members are friendly to each other. 

Members are committed to maintaining close interpersonal relationships. 

Members are committed to group goals and success. 

Social Norm 

People who influence my behavior think that 1 should use the system. 

People who are important to me think that 1 should use the system. 

ENVIRONMENTAL UNCERTAINTY 

Our environment has a high operational tempo. 

Our environment is continuously changing. 

Environmental changes in our area are difficult to forecast. 

CONTROL VARIABLES 

System Voluntariness 

My use of the system is voluntary. 

Using the system is not compulsory to my job. 

Given the choice 1 would choose to use the Collaborative System 1 now use. 

Previous Training, Before Using the System 1 Received: 

No Training 

Classroom Training 

Online Training 

One-on-One Training 

Hands-on Training 

System Customization 

The collaborative system we use adapts to our business processes, rules, and practices. 

The collaborative system we use is customized to our specific needs. 

Personal Habits 

Using a collaborative system has become a habit for me. 

Using my existing system has become natural for me. 

DEMOGRAPHIC DATA 

How many months has your group been using a Collaborative System? 

How many years of work experience do you have? 

What is your age? 

Please provide your email address if you would like a copy of the results of this study. 

Gender: 

What is your educational level? 

What is your current position? 

1 consider myself a{n) user. 
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